Quantization to speedup approximate nearest neighbor search
نویسندگان
چکیده
Abstract The quantization-based approaches not only are the effective methods for solving problems of approximate nearest neighbor search, but also effectively reduce storage space. However, many usually employ fixed nprobes to search process each query. This will lead extra query consumption. Additionally, we observed that as number points in cluster center product quantization increases, cost increases. To address this issue, propose an acceleration strategy based on IVF-HNSW framework further speed up process. involves introducing adaptive termination condition queries and reducing data accessed by building HNSW results. Through extensive experiments, have shown our proposed method significantly accelerates
منابع مشابه
Composite Quantization for Approximate Nearest Neighbor Search
This paper presents a novel compact coding approach, composite quantization, for approximate nearest neighbor search. The idea is to use the composition of several elements selected from the dictionaries to accurately approximate a vector and to represent the vector by a short code composed of the indices of the selected elements. To efficiently compute the approximate distance of a query to a ...
متن کاملApproximate Nearest Neighbor Search by Residual Vector Quantization
A recently proposed product quantization method is efficient for large scale approximate nearest neighbor search, however, its performance on unstructured vectors is limited. This paper introduces residual vector quantization based approaches that are appropriate for unstructured vectors. Database vectors are quantized by residual vector quantizer. The reproductions are represented by short cod...
متن کاملTransformed Residual Quantization for Approximate Nearest Neighbor Search
The success of product quantization (PQ) for fast nearest neighbor search depends on the exponentially reduced complexities of both storage and computation with respect to the codebook size. Recent efforts have been focused on employing sophisticated optimization strategies, or seeking more effective models. Residual quantization (RQ) is such an alternative that holds the same property as PQ in...
متن کاملFast Approximate Nearest-Neighbor Search with k-Nearest Neighbor Graph
We introduce a new nearest neighbor search algorithm. The algorithm builds a nearest neighbor graph in an offline phase and when queried with a new point, performs hill-climbing starting from a randomly sampled node of the graph. We provide theoretical guarantees for the accuracy and the computational complexity and empirically show the effectiveness of this algorithm.
متن کاملApproximate Nearest Neighbor Search in $\ell_p$
We present a new locality sensitive hashing (LSH) algorithm for c-approximate nearest neighbor search in lp with 1 < p < 2. For a database of n points in lp, we achieve O(dn ) query time and O(dn + n) space, where ρ ≤ O((ln c)/c). This improves upon the previous best upper bound ρ ≤ 1/c by Datar et al. (SOCG 2004), and is close to the lower bound ρ ≥ 1/c by O’Donnell, Wu and Zhou (ITCS 2011). T...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
ژورنال
عنوان ژورنال: Neural Computing and Applications
سال: 2023
ISSN: ['0941-0643', '1433-3058']
DOI: https://doi.org/10.1007/s00521-023-08920-3